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Abstract — Sparse representations have emerged as a powerful 
tool in signal and information processing, culminated by the 
success of new acquisition and processing techniques such as 
Compressed Sensing (CS). Fusion frames are very rich new signal 
representation methods that use collections of subspaces instead 
of vectors to represent signals. This work combines these exciting 
fields to introduce a new sparsity model for fusion frames. Signals 
that are sparse under the new model can be compressively 
sampled and uniquely reconstructed in ways similar to sparse 
signals using standard CS. The combination provides a promising 
new set of mathematical tools and signal models useful in a 
variety of applications. With the new model, a sparse signal has 
energy in very few of the subspaces of the fusion frame, although 
it does not need to be sparse within each of the subspaces it 
occupies. This sparsity model is captured using a mixed £1/^2 
norm for fusion frames. 

A signal sparse in a fusion frame can be sampled using very 
few random projections and exactly reconstructed using a convex 
optimization that minimizes this mixed £1 II2 norm. The provided 
sampling conditions generalize coherence and RIP conditions 
used in standard CS theory. It is demonstrated that they are 
sufficient to guarantee sparse recovery of any signal sparse in 
our model. Moreover, a probabilistic analysis is provided using 
a stochastic model on the sparse signal that shows that under 
very mild conditions the probability of recovery failure decays 
exponentially with increasing dimension of the subspaces. 

Index Terms — Compressed sensing, l\ minimization, ^1,2- 
minimization, sparse recovery, mutual coherence, fusion frames, 
random matrices. 

I. Introduction 

COMPRESSED Sensing (CS) has recently emerged as a 
very powerful field in signal processing, enabling the 
acquisition of signals at rates much lower than previously 
thought possible HI, ||2l- To achieve such performance, CS 
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exploits the structure inherent in many naturally occurring 
and man-made signals. Specifically, CS uses classical signal 
representations and imposes a sparsity model on the signal of 
interest. The sparsity model, combined with randomized linear 
acquisition, guarantees that non-linear reconstruction can be 
used to efficiently and accurately recover the signal. 

Fusion frames are recently emerged mathematical structures 
that can better capture the richness of natural and man-made 
signals compared to classically used representations [3J. In 
particular, fusion frames generalize frame theory by using 
subspaces in the place of vectors as signal building blocks. 
Thus signals can be represented as linear combinations of 
components that lie in particular, and often overlapping, signal 
subspaces. Such a representation provides significant flexibil- 
ity in representing signals of interest compared to classical 
frame representations. 

In this paper we extend the concepts and methods of Com- 
pressed Sensing to fusion frames. In doing so we demonstrate 
that it is possible to recover signals from underdetermined 
measurements if the signals lie only in very few subspaces of 
the fusion frame. Our generalized model does not require that 
the signals are sparse within each subspace. The rich structure 
of the fusion frame framework allows us to characterize 
more complicated signal models than the standard sparse or 
compressible signals used in compressed sensing techniques. 
This paper complements and extends our work in |4 |. 

Introducing sparsity in the rich fusion frame model and 
introducing fusion frame models to the Compressed Sensing 
literature is a major contribution of this paper We extend 
the results of the standard worst-case analysis frameworks in 
Compressed Sensing, using the sampling matrix null space 
property (NSP), coherence, and restricted isometry property 
(RIP). In doing so, we extend the definitions to fusion NSP, 
fusion coherence and fusion RIP to take into account the 
differences of the fusion frame model. The three approaches 
provide complementary intuition on the differences between 
standard sparsity and block-sparsity models and sparsity in 
fusion frame models. We note that in the special case that the 
subspaces in our model all have the same dimension, most 
of our results follow from previous analysis of block-sparsity 
models ID. But in the general case of different dimensions 
they are new. 

Our understanding of the problem is further enhanced by the 
probabilistic analysis. As we move from standard sparsity to 
fusion frame or other vector-based sparsity models, worst case 
analysis becomes increasingly pessimistic. The probabilistic 
analysis provides a framework to discern which assumptions 
of the worst case model become irrelevant and which are 
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critical. It further demonstrates the significance of the angles 
between the subspaces comprising the fusion frame. Although 
our analysis is inspired by the model and the analysis in IJlj, 
the tools used in that work do not extend to the fusion frame 
model. The analysis presented in Sec. |V] is the second major 
contribution of our paper. 

In the remainder of this section we provide the motivation 
behind our work and describe some possible applications. 
Section [III provides some background on Compressed Sensing 
and on fusion frames to serve as a quick reference for the 
fundamental concepts and our basic notation. In Section [III] 
we formulate the problem, establish the additional notation and 
definitions necessary in our development, and state the main 
results of our paper We further explore the connections with 
existing research in the field, as well as possible extensions. In 
Section |IV] we prove deterministic recovery guarantees using 
the properties of the sampling matrix. Section [V] presents the 
probabilistic analysis of our model, which is more appropriate 
for typical usage scenarios. We conclude with a discussion of 
our results. 

A. Motivation 

As technology progresses, signals and computational sens- 
ing equipment becomes increasingly multidimensional. Sen- 
sors are being replaced by sensor arrays and samples are being 
replaced by multidimensional measurements. Yet, modern 
signal acquisition theory has not fully embraced the new com- 
putational sensing paradigm. Multidimensional measurements 
are often treated as collections of one-dimensional ones due to 
the mathematical simplicity of such treatment. This approach 
ignores the potential information and structure embedded in 
multidimensional signal and measurement models. 

Our ultimate motivation is to provide a better understanding 
of more general mathematical objects, such as vector-valued 
data points |8j. Generalizing the notion of sparsity is part of 
such understanding. Towards that goal, we demonstrate that 
the generalization we present in this paper encompasses joint 
sparsity models 0, ifTOl as a special case. Furthermore, it is 
itself a special case of block-sparsity models lillj . tf2l . ||5| , 
|l6l, with significant additional structure. 

B. Applications 

Although the development in this paper provides a general 
theoretical perspective, the principles and the methods we 
develop are widely applicable. In particular, the special case of 
joint (or simultaneous) sparsity has already been widely used 
in radar lfT3l . sensor arrays [14], and MRJ pulse design 1 15 |. In 
these applications a mixed £i/£2 norm was used heuristically 
as a sparsity proxy. Part of our goals in this paper is to provide 
a solid theoretical understanding of such methods. 

In addition, the richness of fusion frames allows the appli- 
cation of this work to other cases, such as target recognition 
and music segmentation. The goal in such applications is 
to identify, measure, and track targets that are not well 
described by a single vector but by a whole subspace. In music 
segmentation, for example, each note is not characterized 
by a single frequency, but by the subspace spanned by the 



fundamental frequency of the instrument and its harmon- 
ics fT6^. Furthermore, depending on the type of instrument 
in use, certain harmonics might or might not be present in 
the subspace. Similarly, in vehicle tracking and identification, 
the subspace of a vehicle's acoustic signature depends on the 
type of vehicle, its engine and its tires |17|. Note that in both 
applications, there might be some overlap in the subspaces 
which distinct instruments or vehicles occupy. 

Fusion frames are quite suitable for such representations. 
The subspaces defined by each note and each instrument or 
each tracked vehicle generate a fusion frame for the whole 
space. Thus the fusion frame serves as a dictionary of targets to 
be acquired, tracked, and identified. The fusion frame structure 
further enables the use of sensor arrays to perform joint source 
identification and localization using far fewer measurements 



than a classical sampling framework. In Sec. III-E we provide 



a stylized example that demonstrates this potential. 

We also envision fusion frames to play a key role in video 
acquisition, reconstruction and compression applications such 
as |18|. Nearby pixels in a video exhibit similar sparsity 
structure locally, but not globally. A joint sparsity model such 
as 191, ifTOl is very constraining in such cases. On the other 
hand, subspace-based models for different parts of an image 
significantly improve the modeling ability compared to the 
standard compressed sensing model. 



C. Notation 

Throughout this paper j|x||p = (J^i 



p > denotes 



the standard £p norm. The operator norm of a matrix A from 



ip into ip is written as 



1^1 



p-yp 



1^x11 



II. Background 

A. Compressed Sensing 

Compressed Sensing (CS) is a recently emerged field in 
signal processing that enables signal acquisition using very 
few measurements compared to the signal dimension, as long 
as the signal is sparse in some basis. It predicts that a signal 



X G 



with only k non-zero coefficients can be recovered 
from only n = 0{klog{N/k)) suitably chosen linear non- 
adaptive measurements, compactly represented by 



y = Ax, y e 



A necessary condition for exact signal recovery of all fc-sparse 
X is that 

Az^Q for all z^O, ||z||o < 2k, 

where the £o 'norm,' ||x||o, counts the number of non-zero 
coefficients in x. In this case, recovery is possible using the 
following combinatorial optimization, 

X = argminxgRN ||x||o subject to y = Ax. 

Unfortunately this is an NP-hard problem |fT9l in general, 
hence is infeasible. 

Exact signal recovery using computationally tractable meth- 
ods can be guaranteed if the measurement matrix A satisfies 
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the null space property (NSP) ll20ll . 11211 . i.e., if for all support 
sets S G {1, . . . , iV} of cardinality at most fc, 

1, 



|hs||i< ^llhlli 



for all h e nullsp(A) \ {0}, 



where hg denotes the vector which coincides with h on the 
index set S and is zero outside S. 

If the coherence of A is sufficiently small, the measurement 
matrix satisfies the NSP and, therefore, exact recovery is 
guaranteed ll22ll . ||8]. The coherence of a matrix A with unit 
norm columns a^, llaJk = 1, is defined as 



max |(ai, a^) 



(1) 



Exact signal recovery is also guaranteed if A obeys a restricted 
isometry property (RIP) of order 2k [Ij, i.e., if there exists a 
constant 62k G (0, 1) such that for all 2fc-sparse signals x 

We note the relation 62k < {k—l)fi, which easily follows from 
Gershgorin's theorem. If any of the above properties hold, then 
the following convex optimization program exactly recovers 
the signal from the measurement vector y. 



X = argmm^gj 



x||i subject to y = Ax. 



A surprising result is that random matrices with sufficient 
number of rows can achieve small coherence and small RIP 
constants with overwhelmingly high probability. 

A large body of literature extends these results to mea- 
surements of signals in the presence of noise, to signals that 
are not exactly sparse but compressible |[T|, to several types 
of measurement matrices 1231 . 1241 . l25l . l26l . Il27l and to 
measurement models beyond simple sparsity l28l . 

B. Fusion Frames 

Fusion frames are generalizations of frames that provide a 
richer description of signal spaces. A fusion frame for is 
a collection of subspaces Wj C M*^ and associated weights 
Vj, compactly denoted by (Wj, -WjO^Li. that satisfies 



N 



A||x| 



^ < 

2 - ^ "J 

i=i 



P,x||^ <B|lx|l2 



for some universal fusion frame bounds < A < B < 00 and 
for all X £ M^^, where Pj denotes the orthogonal projection 
onto the subspace Wj. We use rrij to denote the dimension 
of the jth subspace Wj, j — 1, . . . ,N. A frame is a special 
case of a fusion frame in which all the subspaces Wj are one- 
dimensional (i.e., rrij = 1, j = 1, . . . , N), and the weights Vj 
are the norms of the frame vectors. For finite M and N, the 
definition reduces to the requirement that the subspace sum of 
Wj is equal to M^^. 

The generalization to fusion frames allows us to capture 
interactions between frame vectors to form specific subspaces 
that are not possible in classical frame theory. Similar to 
classical frame theory, we call the fusion frame tight if the 
frame bounds are equal, A ~ B. If the fusion frame has 
Vj — 1, j — 1, . . . , N, we call it a unit-norm fusion frame. In 



this paper, we will in fact restrict to the situation of unit- 
norm fusion frames, since the anticipated applications are 
only concerned with membership in the subspaces and do not 
necessitate a particular weighting. 

Dependent on a fusion frame {yVj,Vj)jLi we define the 
Hilbert space H as 

H = {(x,)f^i :x, e forallj = l,...,iV} 



C 



:,MxN 



(or 



We should point out that depending on the use, x can be 
represented as a very long vector or as a matrix. However, 
both representations are just rearrangements of vectors in the 
same Hilbert space, and we use them interchangeably in the 
manuscript. 

Finally, let e jjAfxmj ^ known but othei-wise 
arbitrary matrix, the columns of which form an orthonormal 
basis for Wj, j = 1, . . . , N, that is UjUj = where I,„^. 
is the rrij x nij identity matrix, and U^Uj — Pj. 

The fusion frame mixed Iq^p norm is defined as 

i/p 




(2) 



where (wj)j^i are the fusion frame weights. Furthermore, for 
a sequence c 
mixed norm 



a sequence c = (cj)j^j^, Cj S M™^ , we similarly define the 



N 

El 



The £g Q-'norm' (which is actually not even a quasi-norm) is 
defined as 

l|x||,,o = #{j : X, ^ 0}, 

independent of q. For the remainder of the paper we use q — 2 
just for the purpose of notation and to distinguish £2,0 from 
the £q vector 'norm'. We call a vector x E H k-sparse, if 

||x||2,o < k. 

III. Sparse Recovery of Fusion Frame Vectors 

A. Measurement Model 

We now consider the following scenario. Let x*^ = 
(x°)jLi G T~L, and assume that we only observe n linear 
combinations of those vectors, i.e., there exist some scalars 
aij satisfying || (aij)f^i||2 = 1 for all j = 1, . . . , such that 
we observe 



N 

y = (y.)r=i= IE' 



(3) 



i=l 



where JC denotes the Hilbert space 

= {{y^YU ■■ y. e for alH = 1, . . . , n}. 
We first notice that (|3]l can be rewritten as 

y = Aix°, where Aj = {a,jlM)i<i<n. i<3<n, 
i.e., Ai is the matrix consisting of the blocks UijlM- 
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B. Reconstruction using Convex Optimization 

We now wish to recover x" from the measurements y. If 
we impose conditions on the sparsity of x", it is suggestive to 
consider the following minimization problem, 

X = ai-gminxe«||x||2,o 

N 

subject to o,ij^j = Yi for alH = 1, . . . , n. 

Using the matrix Aj, we can rewrite this optimization problem 

as 

(Po) X = argminxG«||x||2,o subject to Ajx = y. 

However, this problem is NP-hard [119j and, as proposed in 
numerous publications initiated by 1291 . we prefer to employ 
£i minimization techniques. This leads to the investigation of 
the following minimization problem, 

X = argminxe-H||x||2,i subject to Ajx = y. 

Since we minimize over all x — G H and certainly 

PjXj = Xj by definition, we can rewrite this minimization 
problem as 

(Pi) X = argminxe«|lx||2,i subject to Apx = y, 
where 



fa,„P 



ij^j )l<i<n, l<j<N- 



(4) 



Problem (Pi) bears difficulties to implement since mini- 
mization runs over H. Still, it is easy to see that (Pi) is 
equivalent to the optimization problem 



(^i) (cj), 



argmin^^.gR™, ||(UjCj)^i||2,i (5) 
subject to Ai(UjCj)j ~ y, 



where then x = (UjCj)j^i. This particular form ensures that 
the minimizer lies in the collection of subspaces (yVj)^i 
while minimization is performed over Cj £ for all 

j = 1, . . . , N and J^j ™i ^ hence feasible. 

Finally, by rearranging (|5]l, the optimization problems, in- 
voking the ^o-'norm' and ^i-norm, can be rewritten using 
matrix-only notation as 



(Po) 

and 

(Pi) 
in which 

U(c) = 



c = argmin c||c||2,o subject to Y = AU(c) 
c = argmin c II c|| 2.1 subject to Y = AU(c), 



Y = : el 

\ Yn / 

"^ andy, eM^^ 



IX M 



Hereby, we additionally used that ||UjCj||2 ~ \\cj\\2 by 
orthonormality of the columns of Uj . We follow this notation 
for the remainder of the paper. 



C. Worst Case Recovery Conditions 

To guarantee that (Pi) always recovers the original signal 
x'', we provide three alternative conditions, in line with the 



standard CS Uterature M, ES, El, ES, OOl, EB, 
II22II . Specifically, we define the fusion null space property, the 
fusion coherence, and the fusion restricted isometry propery 
(FRIP). These are fusion frame versions of the null space 
property, the coherence and the RIP, respectively. 

Definition 3.1: The pair (A, (Wj)^i), with a matrix A e 
j^nxAf ^jj^ ^ fusion frame (yVj)jLi is said to satisfy the fusion 
null space property if 



lhs||2,i < ^||h||2,i 



for all hG AA\{0}, 



for all support sets S C {!,..., TV} of cardinality at most 
k. Here J\f denotes the null space {h = (hj)j*^i : hj £ 
M™j , AU(h) = 0}, and denotes the vector which coin- 
cides with h on the index set S and is zero outside S. 

Definition 3.2: The fusion coherence of a matrix A G 
jjnxjv .^jj-jj normalized 'columns' (a^ — a. j)^i and a fusion 
frame (Wj)^i for M^^ is given by 



/i/=A*/(A,(W,)Li) = max| 



(a,,afe) 



where Pj denotes the orthogonal projection onto Wj, j ~ 
l,...,N. 

Definition 3.3: Let A G M"""^ and {Wj)^^-^ be a fusion 
frame for M*^ and Ap as defined in (|4]). The /M.s/on restricted 
isometry constant 6k is the smallest constant such that 



(l-4)||z||^,2 < IIApzll^ s < (l + 4)||z 



2 

2.2 



,zw) G 



, of sparsity 



for all z — (zi, Z2, 
l|z||o < k. 

In the case that the subspaces have the same dimension 
the definitions of fusion coherence and fusion RIP coincide 
with those of block coherence and block RIP introduced in 
lH], |I6|. This is expected, as we discuss in Sec. |III-E[ since 
the fusion sparsity model is a special case of general block 
sparsity models. 

Using those definitions, we provide three alternative recov- 
ery conditions, also in line with standard CS results. We first 
state the characterization via the fusion null space property. 

Theorem 3.4: Let A G M"''^ and (Wj)^i be a fusion 
frame. Then all c = (cj)|ti, Cj G with |jc||2,o < fc are 

the unique solution to (Pi) with Y = AU(c) if and only if 
(A, (yVj)j^i) satisfies the fusion null space property of order 
k. 

While the fusion null space property characterizes recovery, 
it is somewhat difficult to check in practice. The fusion 
coherence is much more accessible for a direct calculation, 
and the next result states a corresponding sufficient condition. 
In the case, that the subspaces have the same dimension the 
theorem below reduces to Theorem 3 in |6| on block-sparse 
recovery. 

Theorem 3.5: Let A G M"^^ with normalized columns 
(aj)^i, let (Wj)^i be a fusion frame in R^' , and let Y G 
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^nxM _ jjjgj.g gxists a solution c° of the system AU(c) = Y 
satisfying 



1 



I|c"||2,o<x(i + m7')- 



(6) 



then this solution is the unique solution of (Pq) as well as of 

Finally, we state a sufficient condition based on the fusion 
RIP, which allows stronger recovery results, but is more 
difficult to evaluate than the fusion coherence. The constant 
1/3 below is not optimal and can certainly be improved, but 
our aim was rather to have a short proof. Furthermore, in the 
case that all the subspaces have the same dimension, the fusion 
frame setup is equivalent to the block-sparse case, for which 
the next theorem is implied by Theorem 1 in |5|. 

Theorem 3.6: Let (A,(yVj)^]^) with fusion frame re- 
stricted isometry constant 52k < 1/3. Then (Pi) recovers all 
fc-sparse c from Y = AU(c). 



D. Probability of Correct Recovery 

Intuitively, it seems that the higher the dimension of the sub- 
spaces yVj, the 'easier' the recovery via the li minimization 
problem |7| should be. However, it turns out that this intuition 
only holds true if we consider a probabilistic analysis. Thus 
we provide a probabilistic signal model and a typical case 
analysis for the special case where all the subspaces have the 
same dimension m — nij for all j. Inspired by the probability 
model in Q, we will assume that on the fc-element support 
set S = supp(U(c)) = {ji, . . . ,jk} the entries of the vectors 
Cj, j G S, are independent and follow a normal distribution. 



U(c), 




nkxM 



(7) 



where X 



{Xf . . . X^)^ e M^'" is a Gaussian random 
vector, i.e., all entries are independent standard normal random 
variables. 

Our probabilistic result shows that the failure probability 
for recovering U(c) decays exponentially fast with growing 
dimension ni of the subspaces. Interestingly, the quantity 9 
involved in the estimate is again dependent on the 'angles' 
between subspaces and is of the flavor of the fusion coherence 
in Def. |3.2| Since the block-sparsity model can be seen as a 
special case of the fusion frame sparsity model, the theorem 
clearly applies also to this scenario. 

Theorem 3.7: Let S C {1, . . . , N} be a set of cardinality k 
and suppose that A G R"'^^ satisfies 



|At,A., 



< a < 1 for all j ^ S. 



(8) 



Let {yVj)jLi be a fusion frame with associated orthogonal 
bases {'Uj)^^i and orthogonal projections {'Pj)f^^, and let 



Y e 



Further, let c, e 



J = 1. 



, N with S = 



supp(U(c)) such that the coefficients on S are given by (|7|, 
and let 9 be defined by 



= 1 



max 



1/2 



Choose (5 e (0, 1 — a^). Then with probability at least 



1 - {N ~k)cjq) I m 1 - fccxp 

the minimization problem (Pi) recovers U(c) from Y = 
AU(c). In particular, the failure probability can be estimated 
by 



N exp — max min < , — > ] m 

Let us note that fT, Section V] provides several mild conditions 
that imply ([8]). We exemplify one of these. Suppose that 
the columns of A G E"^^ form a unit norm tight frame 
with (ordinary) coherence /i < c/y/n. (This condition is 
satisfied for a number of explicitly given matrices, see also 
Q.) Suppose further that the support set S is chosen uniformly 
at random among all subsets of cardinality k. Then with high 
probability ([HJ is satisfied provided k < Can, see Theorem 
5.4 and Section V.A in [7]. This is in sharp contrast to 
deterministic recovery guarantees based on coherence, which 
cannot lead to better bounds than k < C^fn. Further note, that 
the quantity 9 is bounded by k in the worst case, but maybe 
significantly smaller if the subspaces are close to orthogonal. 
Clearly, m can only be varied by changing the fusion frame 
model, and 9 may also change in this case. While, in principle, 
9 may grow with to, typically 9 actually decreases; for instance 
if the subspaces are chosen at random. In any case, since 
always 9 < k, the failure probability of recovery decays 
exponentially in the dimension rn of the subspaces. 

E. Relation with Previous Work 

A special case of the problem above appears when all 
subspaces {Wj)f^i are equal and also equal to the ambient 



space Wj 



for all j. Thus, Pj = Im and the observation 



setup of Eq. ([3| is identical to the matrix product 



Y = AX, where X = 



t,NxM 



(9) 



This special case is the same as the well studied joint sparsity 
setup of im, ID, Uni, iMI, EI m wWch a coUection 
of M sparse vectors in is observed through the same 
measurement matrix A, and the recovery assumes that all the 
vectors have the same sparsity structure. The use of mixed 
£i /^2 optimization has been proposed and widely used in this 
case. 

The following simple example illustrates how a fusion frame 
model can reduce the sampling required compared to simple 
joint sparsity models. Consider the measurement scenario of 
(|9|, rewritten in an expanded form: 

T 



r T 1 
-Yi- 




"II 1 






ai a2 ... aAT 


T 




.11 1 









where x^.y^ e M and e W^. Using the fusion frames 
notation, X is a fusion frame representation of our acquired 
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signal, which we assume sparse as we defined in Sec. |II-B| 
For the purposes of the example, suppose that only xi and 
X2 have significant content and the remaining components are 
zero. Thus we only focus on the measurement vectors ai and 
a.2 and their relationship, with respect to the subspaces Wi 
and W2 where xi and X2 lie in. 

Using the usual joint sparsity models, would require that ai 
and a2 have low coherence ([T}, even if prior information about 
Wi and provided a better signal model. For example, if we 
know from the problem formulation that the two components 
Xi and X2 lie in orthogonal subspaces Wi -L yV2, we can 
select ai and a2 to be identical, and still be able to recover 
the signal. If, instead, Wi and W2 only have a common 
overlapping subspace we need ai and a2 to be incoherent 
only when projected on that subspace, as measured by fusion 



Standard Sparsity 

I vector in R™ 



coherence, defined in Def. 3.2 and irrespective of the dimen 



sionality of the two subspaces and the dimensionality of their 
common subspace. This is a case not considered in the existing 
literature. 

The practical applications are significant. For example, 
consider a wideband array signal acquisition system in which 
each of the signal subspaces are particular targets of interest 
from particular directions of interest (e.g. as an extremely 
simplified stylized example consider Wi as the subspace of 
friendly targets and yV2 as the subspace of enemy targets 
from the same direction). If two kinds of targets occupy two 
different subspaces in the signal space, we can exploit this in 
the acquisition system. This opens the road to subspace-based 
detection and subspace-based target classification dictionaries. 

Our formulation is a special case of the block sparsity 
problem ifTTl . lfT2l . |l6l, where we impose a particular structure 
on the measurement matrix A. This relationship is already 
known for the joint sparsity model, which is also a special case 
of block sparsity. In other words, the fusion frame formulation 
we examine here specializes block sparsity problems and 
generalizes joint sparsity ones. 

For the special case of fusion frames in which all the 
subspaces Wj have the same dimension, our definition of 
coherence can be shown to be essentially the same (within 
a constant scaUng factor) as the one in when the problem 
is reformulated as a block sparsity one. Similarly, for the same 
special case our definition of the NSP becomes similar to the 
one in |35|. 

We would also like to note that the hierarchy of such sparsity 
problems depends on their dimension. For example, a joint 
sparsity problem with M = 1 becomes the standard sparsity 
model. In that sense, joint sparsity models generalize standard 
sparsity models. The hierarchy of sparsity models is illustrated 
in the Venn diagram of Fig. [T] 

F. Extensions 

Several extensions of this formulation and the work in 
this paper are possible, but beyond our scope. For example, 
the analysis we provide is in the exactly sparse, noiseless 
case. As with classical compressed sensing, it is possible to 
accommodate sampling in the presence of noise. It is also 
natural to consider the extension of this work to sampling 




Fig. 1. Hierarchy of Sparsity Models 

signals that are not fc-sparse in a fusion frame representation 
but can be very well approximated by such a representation. 
(However, see Section [IV-D| ) 

The richness of fusion frames also allows us to consider 
richer sampling matrices. Specifically, it is possible to con- 
sider sampling operators consisting of different matrices, each 
operating on a separate subspace of the fusion frame. Such 
extensions open the use of £1 methods to general vector-valued 
mathematical objects, to the general problem of sampling such 
objects 18J, and to general model-based CS problems ll28l . 

IV. Deterministic Recovery Conditions 

In this section we derive conditions on c° and A so that c° 
is the unique solution of (Pq) as well as of (Pi). Our approach 
uses the generalized notions of null space property, coherence 
and the restricted isometry property, all commonly used mea- 
sures of morphological difference between the vectors of a 
measuring matrix. 

A. Fusion Null Space Property 



We first prove Thm. 3.4 



which demonstrates that the 
fusion NSP guarantees recovery, similarly to the standard CS 
setup II20I . This notion will also be useful later to prove 
recovery bounds using the fusion coherence and using the 
fusion restricted isometry constants. 

Proof of Theorem \3.4\ Assume first that the fusion NSP 
holds. Let c° be a vector with ||c"||2.o < k, and let be an 
arbitrary solution of the system AU(c) ~ Y, and set 



Letting S denote the support of c*^, we obtain 



|C"1|2,1 



> 



I|csll2,i ^■ 
l|hs<^||2,i 



:S<=I|2,1 - 

hs||2,i- 



|c"sl|2,l 
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This term is greater than zero for any h 7^ provided that 

llhs=||2.i > ||hs||2,i (10) 



or, in other words. 



il|h||2,i> 



(11) 



which is ensured by the fusion NSP. 

Conversely, assume that all vectors c with ||c||o < k are 
recovered using (Pi). Then, for any h. ^ M \ {0} and any 
5 C {1, . . . , with 15*1 < fc, the fc-sparse vector hg is the 
unique minimizer of ||c||2.i subject to Ac = Ahg. Further, 
observe that A(— hgc) = A(hs) and — hgc ^ hg, since h e 
A/'\{0}. Therefore, ||h5||2,i < ||h5c||2,i, which is equivalent 
to the fusion NSP because h was arbitrary. ■ 

B. Fusion Coherence 

The fusion coherence is an adaptation of the coherence 
notion to our more complicated situation involving the angles 
between the subspaces generated by the bases U^, j — 
1, . . . , A^. In other words, here we face the problem of recovery 
of vector-valued (instead of scalar-valued) components and our 
definition is adapted to handle this. 

Since the P^'s are projection matrices, we can also rewrite 
the definition of fusion coherence as 



max 



|l/2 



with Ainax denoting the largest eigenvalue, simply due to the 
fact that the eigenvalues of PfcPjPfc and PjP/c coincide. 
Indeed, if A is an eigenvalue of PjPfc with corresponding 



eigenvector v then P^P^P^ 



XPkV. 



Since P| 



k 



this implies that PfcPjPfc(Pfcu) = APfcU so that A is an 
eigenvalue of Pj-P^Pfe with eigenvector V^v. Let us also 
remark that |Aniax(PjPfc)r^^ equals the largest absolute value 
of the cosines of the principle angles between Wj and Wk- 
Before we continue with the proof of the recovery condition 



in Theorem 3.5 let us for a moment consider the following 



special cases of this theorem. 

a) Case M — 1: In this case the projection matrices 
equal 1, and hence the problem reduces to the classical 
recovery problem Ax = y with x e and y G M". 
Thus our result reduces to the result obtained in [36 1, and the 
fusion coherence coincides with the commonly used mutual 
coherence, i.e., — maxj^^ |(aj-,afe)|. 

b) Case Wj = M*^ for all j: In this case the problem 
becomes the standard joint sparsity recovery. We recover a 
matrix X° G M^^^^ with few non-zero rows from knowledge 
of AX° S M"^*^, without any constraints on the structure 
of each row of X°. The recovered matrix is the fusion 
frame representation of the sparse vector and each row j 
represents a signal in the subspace Wj (the general case has 
the constraint that X" is required to be of the form U(c°)). 
Again fusion coherence coincides with the commonly used 
mutual coherence, i.e., /i/ = maxj^^ [(a^ , aj;)|. 

c) Case Wj -L Wk for all j, k: In this case the fusion 
coherence becomes 0. And this is also the correct answer, since 
in this case there exists precisely one solution of the system 
AU(c) = Y for a given Y. Hence Condition (|6| becomes 
meaningless. 



d) General Case: In the general case we can consider 
two scenarios: either we are given the subspaces (Wj)j or 
we are given the measuring matrix A. In the first situation 
we face the task of choosing the measuring matrix such that 
is as small as possible. Intuitively, we would choose the 
vectors {sLj)j so that a pair (aj,afe) has a large angle if the 
associated two subspaces {yVj,Wk) have a small angle, hence 
balancing the two factors and try to reduce the maximum. 
In the second situation, we can use a similar strategy now 
designing the subspaces iWj)j accordingly. 



For the proof of Theorem 3.5 we first derive a reformulation 
of the equation AU(c) = Y. For this, let Pj denote the 
orthogonal projection onto Wj for each j = 1,...,N, set 
Ap as in Q and define the map (/Jfe : K''^^^ ^ M'^*^, fc > 1 
by 



iPkCZ) = (pk 



(Zl . . .Zfc) 



i.e., the concatenation of the rows. Then it is easy to see that 
AU(c)=Y ^ Ap</pjv(U(c)) = (^„(Y). (12) 



We now split the proof of Theorem |3.5| into two lemmas, 
and wish to remark that many parts are closely inspired by 
the techniques employed in ll36l . OTl . We first show that c" 
satisfying (|6]l is the unique solution of (Pi). 

Lemma 4.1: If there exists a solution U(c") e jj^vxAf 
the system AU(c) = Y with c'' satisfying (|6|, then c'' is the 
unique solution of (Pi). 

Proof: We aim at showing that the condition on the fusion 
coherence implies the fusion NSP. To this end, let h e A/^\{0}, 
i.e., AU(h) = 0. By using the reformulation ( [T2] i, it follows 
that 

Ap(^^(U(h)) = 0. 



This implies that 



A^Ap(^w(U(h)) = 0. 



Defining a^ by a^ — {aij)i for each j, the previous equality 
can be computed to be 

((a,,afc)PjPfc),fc(^jv(U(h)) = 0. 

Recall that we have required the vectors a^ to be normalized. 
Hence, for each j. 



Since ||U,-h 



i"i II 2 



= (aj,afc) PjPfeUfeh/; 

|hj||2 for any j, this gives 



|h,||2 < 5]|(a„a,)|-||P,P,.||2^2||hfc|, 
< /i/(l|h||2,i-||h,||2), 



which implies 



I|h,|l2<(l + M7'r'l|h|l24 
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Thus, we have 



1S||2,1 



< 



l|c°||2,o-(l + A^7') 



h||2,i 



2,1- 



Concluding, (|6]l and the fusion null space property show that 
h satisfies ( fTTj ) unless h = 0, which implies that c*' is the 
unique minimizer of (Pi) as claimed. ■ 
Using Lemma [4T| it is easy to show the following lemma. 



Lemma 4.2: If there exists a solution U(c") e 



pNxM 



of 



the system AU(c) = Y with c° satisfying (|6|, then c" is the 
unique solution of (Po)- 

Proof: Assume satisfies (|6| and AU(c'') = Y. Then, 
by Lemma 4.1 it is the unique solution of (Pi). Assume there 



is a c satisfying AU(c) ~ Y such that ||c||2,o < 



|C°|!2,0. 



Then c also satisfies (|6]l and again by Lemma 4.1 c is also 



the unique solution to (Pi). But this means that c = c" and 
c° is the unique solution to (Pq). ■ 



We observe that Theorem |3.5| now follows immediately 
from Lemmas 14.11 and 14.21 



C. Fusion Restricted Isometry Property 

Finally, we consider the condition for sparse recovery using 
the restricted isometry property (RIP) of the sampling matrix. 
The RIP property on the sampling matrix, first introduced 
in H], complements the null space propery and the mutual 



coherence conditions. Definition 3.3 generalizes it for the 



fusion frame setup. Informally, we say that (A, (Wj)jLi) 
satisfies the fusion restricted isometry property (FRIP) if 5k is 
small for reasonably large k. Note that we obtain the classical 
definition of the RIP of A if Af = 1 and all the subspaces 
Wj have dimension 1. Using this property we can prove 
Theorem 13.61 



Proof of Theorem 3.6 



The proof proceeds analogously 
to the one of Theorem 2.6 in [26], that is, we establish the 
fusion NSP. The claim will then follow from Theorem 13.41 
Let us first note that 



|(ApU,Apv)| < 4||u|l2,2||v|l2.2 



pMN 



for all u = (ui, . . . ,UAr),v = (vi,...,VAr) e 
Uj,Vj e M*^, with suppu = {j : 7^ 0} n suppv = 
and ||u||2.o + ||v||2,o ^ k. This statement follows completely 
analogously to the proof of Proposition 2.5(c) in lf32l . see also 

lEzj, m- 

Now let h e TV = {h : AU(h) = 0} be given. Using the 
reformulation ([12]), it follows that 

Ap(^Ar(U(h)) =0. 

In order to show the fusion NSP it is enough to consider 
an index set 5*0 of size k of largest components ||hj||2, i.e., 
Ilh,|l2 > ||h,||2 for all j e So, z € = {1, ... ,7V} \ So- 
We partition Sq into index sets 81,82, ■■ ■ of size k (except 
possibly the last one), such that 5*1 is an index set of largest 
components in 8q, 82 is an index set of largest components 
in {80 U 8oy, etc. Let hg. be the vector that coincides with 
h on 8i and is set to zero outside. In view of h E Af 
we have AU(h5^) = AU(-h5j - h^^ - •••). Now set 
z = </37v(U(h)) and zg. (/3Ar(U(hsJ). It follows that 



Ap(z5q) = Ap(— ^-^^ Z5.). By definition of the FRIP we 
obtain 

1 



|hsoll2,2 = l|zSoll2,2< 



1 



|ApZ5„||2.2 



1 

1-Sl. 



Apzs„, Ap( - ^zs^ 



i>i 



< ^EKApZ5„,Ap(-zsJ)| 

i>l 



< 



J2k 



1 



'oll2,2 ■ ^ I 



ZsJ|2,2- 



Using that Sk < 62k and dividing by ||z5q||2.2 yields 



|zSoll2,2 < 



1 - S2k 



By construction of the sets 8i we have ||zj||2 
I Sfes._i W'^'^h for all j e 8i, hence. 



< 



|ZS'J|2,2 



;5:ilz,ii^)^/^<i=llz,_ii2,i. 



The Cauchy-Schwarz inequality yields 

||hse||2,l < ^||h5j|2.2< Y^Ell^S, 



2,1 



< 



"Iz5jl2,l + ||z5sl|2,l)<^||h|l2,l, 



where we used the assumption S2k < 1/3. Hence, the fusion 
null space property follows. ■ 
Having proved that the FRIP ensures signal recovery, our 
next proposition relates the classical RIP with our newly 
introduced FRIP. Let us note, however, that using the RIP 
to guarantee the FRIP does not take into account any prop- 
erties of the fusion frame, so it is sub-optimal — especially if 
the subspaces of the fusion frame are orthogonal or almost 
orthogonal. 

Proposition 4.3: Let A e M"^^ with classical restricted 
isometry constant 6k, that is, 

(l-4)lly||^<||Ay||2<(i + 4)||y|| 

for all fc-sparse y £ M^. Let (yVj)^i be an arbitrary fusion 
frame for M^^. Then the fusion restricted isometry constant Sk 
of (A, (W)f^i) satisfies 4 < 4. 

Proof: Let c satisfy ||c||o < k, and denote the columns 
of the matrix U(c) by Ui, . . . . um- The condition ||c||o < k 
implies that each is fc-sparse. Since A satisfies the RIP of 
order fc with constant 6k, we obtain 

M 



|AU(c)||i2 



M 

El 



Au,||2 < (1 + 4) Ell"' 



i\\2 



(i + 4)||U(c)|||2 = (i + 4)||c||^, 



as well as 



M 



|AU(c)||2 2 = 5]||Au,||2> (i_4)^||u, 



= (l-4)||U(c)||L = (l-4)||c|lL- 
This proves the proposition because AU(c) ~ ApU(c). 
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D. Additional Remarks and Extensions 



Of course, it is possible to extend the proof of Theorem 3.6 
in a similar manner to ll37l . ||38J . ||39J such that we can 
accommodate measurement noise and signals that are well 
approximated by sparse fusion frame representation. We state 
the analog of the main theorem of Vil\ without proof. 

Theorem 4.4: Assume that the fusion restricted isometry 
constant 52k of (A, (yVj)^i) satisfies 



52k < A 



/2 - 1 ss 0.4142. 



For X G H, let noisy measurements Y = Ax+?; be given with 
||'7||2 < £■ Let c'^ be the solution of the convex optimization 
problem 

min ||c||2,i subject to || AU(c) - Y||2,2 < ?7, 
and set x'^ = U(c#). Then 



r#\ 



< Cir] + C2- 



X 2,1 



where Xk is obtained from x be setting to zero all components 
except the k largest in norm. The constants Ci , C2 > only 
depend on d2k (or rather on A — d2k)- 

V. Probabilistic Analysis 

A. General Recovery Condition 

We start our analysis by deriving a recovery condition on 
the measurement matrix and the signal which the reader might 
want to compare with [40|, |41|. Given a matrix X e R^^^, 
we let sgn(X) e jj^^xa^ denote the matrix which is generated 
from X by normalizing each entry Xji by the norm of the 
corresponding row Xj . . More precisely, 

r ^ 

sgn(X)j, = <^ 







if 
if 



IX, 



IX, 



|2^0, 

I2 =0. 



Column vectors are defined similarly by X. i. 

Under a certain condition on A, which is dependent on the 
support of the solution, we derive the result below on unique 



recovery. To phrase it, let {yVj)j^i be a fusion frame with 
associated orthogonal bases {Uj)jLi and orthogonal projec- 
tions {Pj)^^i, and recall the definition of the notion U(c) in 



Section 
{1,..., 



and 



III 



Then, for some support set S — {ji, . . . ,j\s\} ^ 



TV] of U(c), we let 
As = (A.^ 



j\s\) € 



inx |S| 



U(c)s = 



3\s\ J\s\ 



xM 



Before stating the theorem, we wish to remark that its proof 
uses similar ideas as the analog proof in |7|. We however state 
all details for the convenience of the reader 

Theorem 5.1: Retaining the notions from the beginning of 



this section, we let c, e 



J 



l...,N with S 



supp(c) = {j : Cj 0}. If As is non-singular and there 



exists a matrix H e 



A?H 



such that 

sgn(U(c)5) 



(13) 



and 



|H^A.,^||2<1 forallj^S*, 



(14) 



then U(c) is the unique solution of (Pi). 

Proof: Let G , j = 1 . . . , be a solution of 
Y AU(c), set S = supp(U(c)), and suppose A5 is non- 
singular and the hypotheses ( [T3] l and ( [T4] i are satisfied for 



some matrix H e 



prix Af 



Let c' e 



1 . 



with 



c'j^^ 7^ for some jo be a different set of coefficient vectors 
which satisfies Y = AU(c'). To prove our result we aim to 
establish that 

(15) 



c 2,1 < C 2,1. 



We first observe that 



||C||2,1 = ||U(C)||2,1 = ||U(c)s||2,l 

= tr[sgn(U(c)s)(U(c)s)n. 

Set S' = supp(U(c')), apply ( [T3] l, and exploit properties of 
the trace, 

||c||2,i = tr[A^H(U(c)5)^] -tr[(AU(c))^H] 
= tr[(AU(c'))^H] 

= tr [(AU(c'5))^H] +tr [(AU(c'^.))^H] . 
Now use the Cauchy-Schwarz inequality to obtain 

||C||2,1 < 5]||(U(c')5),,.||2||(H^As).,,||2 

+ J2 ll(U(c')50..-||2||(H^Asc).,,|i2 

jes- 

< max||(H^As).,,||2||c'5||2,i 
+ max||(H^Asc).j||2||c;j.||2,i 



< \\csh,i 



C 2,1- 



The strict inequality follows from ||c'gc||i > 0, which is true 
because otherwise c' would be supported on S. The equality 
AU(c) = AU(c') would then be in contradiction to the 
injectivity of A5 (recall that c 7^ c'). This concludes the proof. 



The matrix H exploited in Theorem 5.1 might be chosen 



as 



H = (A^5)^sgn(U(c)5) 



to satisfy ( [T3] l. This particular choice will in fact be instru- 
mental for the average case result we are aiming for For now, 
we obtain the following result as a corollary from Theorem 

ED 

Corollary 5.2: Retaining the notions from the beginning 
of this section, we let e M™j , j — 1 . . . ,N, with 
S = supp(U(c)). If Ag is non-singular and 

||sgn(U(c)s)^A^5A.,,||2 < 1 for all j ^ S, 

then U(c) is the unique solution of (Pi). 

For later use, we will introduce the matrices XJj e 
defined by 



(16) 



pAIxNm 



(0 



Mxr 



•10 



Mxr 



10 



Mxt 



10 
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where XJj is the jth block. For some h = {bi, . . . ,bk)^ E 
we can then write U(c)^b as 



Lemma 5.4: Let b = (6i,...,6fe)^ G M'' and S = 
{ji,-..,Jfe}C{l,...,iV}. Then 



(17) 



Using the above and the probabilistic model in Sec. III-D 



to prove our main probabilistic result. Theorem 3.7 



B. Probability of Sparse Recovery for Fusion Frames 



The proof of our probabilistic result, Theorem 3.7 



IS 

developed in several steps. A key ingredient is a concentration 
of measure result: If / is a Lipschitz function on with 
Lipschitz constant L, i.e., \f{x) — f{y)\ < L\\x — y\\2 for 
all x,y e M^, X is a ii'-dimensional vector of independent 
standard normal random variables then P2l eq. (2.35)] 

V{\f{X) - Ef{X)\ > u) < 2e-"'/(2L=^) foj. ^jj ^ > q. 

(18) 

Our first lemma investigates the properties of a function 
related to ([TtJi that are needed to apply the above inequality. 

Lemma 5.3: Let b — G M*^ and S — 

{ji, . . . , jfc} C {1, . . . , A^}. Define the function / by 



/(X) = ||^&,U,,X|l2, Xe 



Then the following holds. 

(i) / is Lipschitz with constant || X]f=i ^fUjf || 2-5.2- 

(ii) For a standard Gaussian vector X e M^™ we have 

E[/(X)] < y^^||b||2. 

Proof: The claim in (i) follows immediately from 

k k 

|/(X)-/(Y)| = |||^6,U,,X||2-||^5,U,,Y||2 



£=1 



< 



< 



f k \ 

^6,U,J (X-Y) 



112^2 



IX-Yll 



It remains to prove (ii). Obviously, 



(E/(X))2 < E[/(X)2]=E 



M k 



M k 

= E E &£6rE[(U,,X),(U,,,X),]. 

i=i £,£'=1 

Invoking the conditions on X, 

AI k k 



E[/(X)] = 



< 



5]5]&?E[(U,.X),]2 = ^62|,u 



jt IIF 



1=1 £=1 

m\\h\\l 



Next we estimate the Lipschitz constant of the function / 
in the previous lemma. 



k 

E 

£=1 



&AJ|2^2< ||b|U /l + niax V A^ax(P,:P,)i/ 



Proof: First observe that 



EW-Jl2^2 = ||(^6,U,J^(^6,U 



1/2 
Jiyil2-i-2 



E b£b£,^l^,, 

e,£'=i 



1/2 

2-i.2- 



Since 



^ 6,VUj,U,„ 112^2 < ||b||^||(UfU,),,,es||2^2, 



•£' = 1 



it follows that 



k 

E 

^=1 



112^2 < llb|Ull(UfU,).,,es|l2^'2- (19) 



Next, 



|(UfU,).,,e5||2^2 < max^ ||Uf U,||2^2 (20) 



l + max l|UfU,||2^2. 



By definition of the orthogonal projections 



|UfU,||2^2 



= ||(u-u,)-u-u,||^^, 

= ||UfU,UjU.||^^^, = | 

— \ CP 13 \^/'^ 



■Pi-Pill2^-2 



Combining with (|20| 



(Uf Uj)ijgs||2-i-2 < 1 + max A 

max V-"- 7 / 

2^5 ^ — ' 



/2 



(21) 

The lemma now follows from ([19]), pO] ), and ( |2T| . ■ 

Now we have collected all ingredients to prove our main 
result. Theorem 1 3. 7 1 

Proof Denote b^^) = {b[^\ . . .,b[^^)^ = At,A.^ t^'' 



for all j ^ S and choose S e (0, 1 — a ). By Corollary 5.2 the 
probability that the minimization problem (Pi) fails to recover 
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U(c) from Y = AU(c) can be estimated as 

P (max||sgn(U(c)s)^b(^)||2 > 1 



< 



maxll ^6y'u,,X|l2 > 7(1^ 
\-' i=i 



max 

1=1,. ...k 



jis \ i=l ) 

k 

+ J2P{\\X,\\l<{l-S)m). 
e=i 

Since is a standard Gaussian vector in B3l Corollary 

3] gives 

P {\\Xe\\l < (1 - S)m) < exp{-S'^m/4). 
Furthermore, the concentration inequality (fTSjl combined with 



Lemmas 5.3 and Lemma 5.4 yields 



k \ 

1=1 I 

k 



< exp 

< exp 

< exp I — 



+(\/r^- I|Mj')||2)V^) 



-\\b^^'>\\2)'^m 

(\/r^- ||6(j)||2)'m 
2\\bU)\\p 



— S — aYm 

Combining the above estimates yields the statement of the 
Theorem. ■ 

VI. Conclusions and Discussion 

The main contribution in this paper is the generalization 
of standard Compressed Sensing results for sparse signals to 
signals that have a sparse fusion frame representation. As we 
demonstrated, the results generalize to fusion frames in a very 
nice and easy to apply way, using the mixed ^2/^1 norm. 

A key result in our work shows that the structure in fusion 
frames provides additional information that can be exploited in 
the measurement process. Specifically, our definition of fusion 
coherence demonstrates the importance of prior knowledge 
about the signal structure. Indeed, if we know that the signal 
lies in subspaces with very little overlap (i.e., where ||PjPfe||2 



matrix (i.e., |(aj,afe)| in the same definition) and maintain a 
low fusion coherence. This behavior emerges from the inherent 
structure of fusion frames. 

The emergence of this behavior is evident both in the 
guarantees provided by the fusion coherence, and in our 
average case analysis. Unfortunately, our analysis of this 
property currently has not been incorporated in a tight ap- 
proach to satisfying the Fusion RIP property, as described in 



Section IV-C While an extension of such analysis for the RIP 



is small in Definition 3.2 1 we can relax the requirement on 
the coherence of the corresponding vectors in the sampling 



guarantees is desirable, it is still an open problem. 

Our average case analysis also demonstrates that as the 
sparsity structure of the problem becomes more intricate, the 
worst case analysis can become too pessimistic for many prac- 
tical cases. The average case analysis provides reassurance that 
typical behavior is as expected; significantly better compared 
to the worst case. Our results corroborate and extend similar 
findings for the special case of joint sparsity in Q. 
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